Press Release Summary: The current procedure for analyzing the financial bailout is flawed. It is important that our leaders follow an accurate method of forecasting. In uncertain times, expertise, confidence, and expert opinions may still lead to inaccurate predictions. Dr. J. Scott Armstrong and Dr. Kesten Green, forecasting experts, reveal weaknesses in the current forecasting methods of the economy.
Press Release Body: Assume that you are considering investing $700,000 and that this is a large sum for you. Experts give you the names of two reputable investment houses of long standing. When you visit the first of these, Benjamin Company, Inc., the manager recommends Bailout Bonds as an excellent investment for you. He gives you what sound like good reasons for his recommendation and reassures you that he is very confident in his advice, which is based on the analysis of top experts. He urges you to act swiftly to avoid missing out. As a cautious investor, you visit the other investment house that was recommended to you, Franklin Company, Inc., their manager advises you that Bailout Bonds would be a poor investment and that you would be better off holding on to your money than putting such a large some into such a speculative investment. Like Benjamin’s manager, Franklin’s manager provides good reasons and she too is confident in her advice. Given the conflicting advice, you decide to check the track record of each of these investment houses. Fortunately, numerous academic studies have been published on the accuracy of each firm’s investment recommendations since 1930. Each firm retains the advice of some of the best experts in the world. You find that on average, the Benjamin experts have been right for 50% of their forecasts – and wrong on 50%. Interestingly, the Franklin experts have exactly the same performance record. What would you do? Our guess on what most small business people would do is skip this investment opportunity. The year 1930 is roughly the date of the first studies on the value of expert forecasts that relate to complex and uncertain situations. Contrary to popular opinion, the findings are consistent with our Benjamin versus Franklin story. For example, Philip E. Tetlock’s 2005 book, Expert Political Judgment describes how he recruited 284 people whose professions included “commenting or offering advice on political and economic trends.” He asked them to forecast the probability that various situations. By 2003, he had accumulated 82,361 forecast. The conclusions from the studies are that: 1) expert opinions are useless for forecasting related to complex and uncertain situations. 2) expertise does not help; College students do as well as seasoned experts at such forecasting. 3) experts’ statements about confidence have virtually no value. Indeed, if you put a group of experts in a room and have them make forecasts, their confidence goes up rapidly, but this has no relationship to accuracy. The country faces a similar problem. But our leaders in Washington are not debating about their own investments. Instead, they are thinking about how to spend other people’s money. They have provided no scientific basis for their decision and they show no awareness of how one should properly approach such a forecasting problem. There are ways to study this problem, but they should not be done in a rush, and they should not be done in group meetings. We don’t know what will happen, but we do know what procedures to use to obtain scientific forecasts of the outcomes of various plans. Researchers in the field have been trying to spread the word on scientific (evidence-based) forecasting by making forecasting knowledge easily and freely available to others at http://forecastingprinciples.com The question for our leaders is whether they should invest $700 billion when, despite their confidence, they are completely ignorant of the outcome of the investment plan.
Dr. J. Scott Armstrong. Professor, The Wharton School, University of Pennsylvania Dr. Kesten C. Green. Business and Economic Forecasting Unit, Monash University.
Web Site: http://publicpolicyforecasting.com
Contact Details: 700 Jon M. Huntsman Hall 3730 Walnut Street Wharton School University of Pennsylvania Philadelphia, PA 19104
2022
